Method, apparatus and recording medium for image processing

ABSTRACT

The sense of contrast perceived by a viewer of an image is quantified and adequate image processing is carried out on image data based on the sense of contrast. Contrast-sense quantification means generates unsharp image data of the image data and then generates a histogram of the unsharp image data. Since the histogram of the image data includes lightness information of details of the image, a distribution width thereof does not represent the contrast perceived by the viewer of the image as a whole. However, since the histogram of the unsharp image data excludes information of the details, a distribution width of the unsharp image data represents the contrast of the overall image. The distribution width of the histogram of the unsharp image data is found as the sense of contrast and input to processing means. In the processing means, tone conversion processing is carried out on the image data by changing a tone conversion LUT based on the sense of contrast, and processed image data are obtained.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a divisional application of application Ser.No. 09/639,804 filed on Aug. 17, 2000, now U.S. Pat. No. 7,068,328. Thepresent application also claims priority to Japanese Patent ApplicationNo. 230731/1999, filed Aug. 17, 1999 and Japanese Patent Application No.173279/2000, filed Jun. 9, 2000, the entire contents which are herebyincorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and an apparatus forquantifying a sense of contrast of an image perceived by a viewer andfor carrying out image processing on image data based on the sense ofcontrast having been quantified. The present invention also relates to acomputer-readable recording medium storing a program to cause a computerto execute the image processing method.

2. Description of the Related Art

Reproduction of digital image data obtained by a digital camera or byreading an image recorded on a film as a softcopy on a display device ora hardcopy such as a print has been carried out. In the case wheredigital image data are reproduced as has been described above, variouskinds of image processing such as tone processing and frequencyprocessing has been carried out on image data so as to cause the qualityof the reproduced image to be as high as that of a print generated froma negative film.

For example, a histogram of image data is generated and contrast of animage represented by the image data is found by a distribution width ofthe histogram. By correcting a tone curve for converting a tone of theimage data based on the contrast having been found, the image data areconverted so that a tone does not become flat or noise becomesinconspicuous. Various kinds of such image processing methods have beenproposed (Japanese Unexamined Patent Publication No. 6(1994)-253176, forexample). The contrast here means a ratio of a dark portion of an imageto a light portion of the image. Therefore, the contrast of an image canbe judged by the distribution width of a histogram. If the histogramdistribution width is wide, the image has sharp contrast while the imagehas weak contrast if the distribution width is narrow. For example, animage obtained by photographing in fine weather has a wide histogramreflecting a dark portion in the shade and a light portion in thesunlight. If an image is obtained by photographing in cloudy weather,the histogram becomes narrow due to poor distinction between thesunlight and shade.

Another image processing method for quantifying a sense of sharpness orgraininess of an image perceived by a viewer and for changing thecontent of image processing based on the sense of contrast having beenquantified in order to generate a preferable image has also beenproposed (Japanese Unexamined Patent Publication No. 7(1995)-193766).

The histogram generated from image data has a complex shape since thehistogram includes various kinds of information of all subjects or everydetail in an image. Therefore, information of the contrast perceived bya viewer of the image is buried in the complex shape, and the histogramdoes not necessarily reflect the viewer's sense of contrast. Forexample, in the case of an image having a human face as a subjectthereof, a viewer of the image perceives the contrast only in the faceand not for subjects other than the face in the image. However, since ahistogram generated from image data includes information regarding thesubjects other than the face, this histogram does not reflect thecontrast perceived by the viewer. Therefore, if image processing iscarried out on the image data based on such a histogram, a processedimage desired by the viewer is not necessarily obtained.

The sense of contrast perceived by a viewer becomes different betweenthe cases of sharp colors and dull colors included in an image. Forexample, an image having sharp colors therein is perceived as an imagehaving sharp contrast while an image including colors which are not sosharp is perceived as an image having weak contrast. Therefore, sincethe sense of contrast becomes different due to colors included in animage, image processing considering the colors of an image needs to becarried out.

The present invention has been conceived based on consideration of theabove problems. An object of the present invention is therefore toprovide a method and an apparatus for quantifying a sense of contrastperceived by a viewer of an image and for carrying out adequate imageprocessing on the image based on the sense of contrast, and also toprovide a computer-readable recording medium storing a program to causea computer to execute the image processing.

Another object of the present invention is to provide a method and anapparatus for carrying out adequate image processing on an image byusing color information of the image, and also to provide acomputer-readable recording medium storing a program to cause a computerto execute the image processing.

SUMMARY OF THE INVENTION

When a viewer observes an image and judges the contrast thereof, thecontrast is judged based on not only a shade difference of all subjectsas a whole but also information not reflected in a histogram, such as ashade difference of the entire image, distributions of dark and lightportions therein, a distribution of lightness only in a subject ofinterest, and the like. The present invention has been conceived bypaying attention to this fact.

In other words, a first image processing method of the present inventionis characterized by the fact that the sense of contrast of an imagerepresented by image data is quantified based on the image data.

The “sense of contrast” herein referred to means a subjective senseregarding contrast of an image perceived by a viewer and not directlyreflected in a histogram of the image itself, such as a shade differenceof the entire image, a distribution of dark and light portions in theimage, and a lightness distribution only in a subject of interest. Morespecifically, the sense of contrast can be quantified based on ahistogram of unsharp image data of the image data, information ofpositions of a light portion and/or a dark portion in an unsharp imagerepresented by the unsharp image data, and a histogram or the likeobtained by multi-resolution image data in each of frequency bandsgenerated by conversion of the image data into multiple resolutions, forexample.

Although the histogram can be generated from the unsharp image datathemselves, the histogram may be generated from unsharp image dataconverted into 32-bit numbers, hexadecimal numbers or octal numbers whenthe image data has 8-bit (256) information, for example.

Furthermore, luminance data and color data representing luminanceinformation and color information of an image may be obtained from theimage data. Unsharp luminance image data and/or unsharp color image datawhich are unsharp image data of the luminance data and/or the color dataare then generated. A luminance histogram and/or a color histogram aregenerated from the unsharp luminance image data and/or the unsharp colorimage data, and based on the luminance histogram and/or the colorhistogram, the sense of contrast may be quantified. The “colorinformation” refers to information representing sharpness of a colorincluded in an image.

When the unsharp color image data are generated, a color histogramrepresenting a two-dimensional frequency distribution of the unsharpcolor image data may be used.

Moreover, as the “position information of a light portion and/or a darkportion”, standard deviation of a distance from the center of an imageto the light portion and/or the dark portion can be used.

A phrase stating “based on the histogram or the like obtained frommulti-resolution image data” refers to the case of quantification of thesense of contrast based on a rough lightness distribution of an imagefound by a low-frequency band image represented by image data at aresolution in a low frequency band generated from image data, and basedon a histogram of a medium-frequency band image or a high-frequency bandimage represented by image data at a resolution of a medium frequencyband or a resolution of a high frequency band generated from the imagedata, for example.

The luminance data and the color data representing the luminanceinformation and the color information of an image may be obtained fromimage data. The luminance data and the color data are then convertedinto multiple resolutions so that multi-resolution luminance image dataand multi-resolution color image data in a plurality of frequency bandsare obtained. The sense of contrast may be quantified based on luminancehistograms and/or color histograms which are histograms of themulti-resolution luminance image data and the multi-resolution colorimage data.

In the first image processing method of the present invention, it ispreferable for image processing to be carried out on the image databased on the sense of contrast.

In this case, it is preferable for the image processing to be at leastone of tone conversion processing, frequency enhancing processing, AEprocessing and chroma conversion processing.

A second image processing method of the present invention ischaracterized by the fact that image processing for changing luminanceinformation of an image is carried out on image data based on colorinformation of the image represented by the image data.

In the second image processing method of the present invention, it ispreferable for the image processing to be carried out on the image dataaccording to the steps of:

obtaining color data representing color information of the image fromthe image data;

generating unsharp image data of the color data;

generating a histogram of the unsharp image data; and

carrying out the image processing on the image data based on thehistogram.

In this case, a histogram representing a two-dimensional frequencydistribution of the unsharp image data is preferably generated.

In the second image processing method of the present invention, it ispreferable for the image processing to be carried out according to thesteps of:

obtaining color data representing color information of the image basedon the image data;

obtaining multi-resolution image data in a plurality of frequency bandsby converting the color data into multiple resolutions;

generating a histogram of multi-resolution data in a lowermost frequencyband out of the multi-resolution image data in the plurality offrequency bands; and

carrying out the image processing on the image data based on thehistogram.

A first image processing apparatus of the present invention ischaracterized by the fact that the apparatus has contrast-sensequantification means for quantifying a sense of contrast of an imagerepresented by image data, based on the image data.

In the first image processing apparatus of the present invention, it ispreferable for the contrast-sense quantification means to comprise:

unsharp image data generating means for generating unsharp image data ofthe image data;

histogram generating means for generating a histogram of the unsharpimage data; and

quantification means for quantifying the sense of contrast based on thehistogram.

In the first image processing apparatus of the present invention, it ispreferable for the contrast-sense quantification means to comprise:

conversion means for obtaining luminance data and color datarepresenting luminance information and color information of the imagefrom the image data;

unsharp image data generating means for generating unsharp luminanceimage data and/or unsharp color image data which are unsharp image dataof the luminance data and/or the color data;

histogram generating means for generating a luminance histogram and/or acolor histogram which are histograms of the unsharp luminance image dataand/or the unsharp color image data; and

quantification means for quantifying the sense of contrast based on theluminance histogram and/or the color histogram.

In the case where the unsharp image data generating means generates theunsharp color image data, it is preferable for the histogram generatingmeans to generate a color histogram representing a two-dimensionalfrequency distribution of the unsharp color image data.

Furthermore, it is preferable for the contrast-sense quantificationmeans to comprise:

unsharp image data generating means for generating unsharp image data ofthe image data; and

quantification means for quantifying the sense of contrast based onposition information of a light portion and/or a dark portion in anunsharp image represented by the unsharp image data.

It is also preferable for the contrast-sense quantification means tocomprise:

multi-resolution conversion means for obtaining multi-resolution imagedata in a plurality of frequency bands by converting the image data intomultiple resolutions;

histogram generating means for generating a histogram of themulti-resolution image data; and

quantification means for quantifying the sense of contrast based on thehistogram.

Furthermore, it is also preferable for the contrast-sense quantificationmeans to comprise:

conversion means for obtaining luminance data and/or color datarepresenting luminance information and/or color information from theimage data;

multi-resolution conversion means for obtaining multi-resolutionluminance data and/or multi-resolution color data in a plurality offrequency bands by converting the luminance data and/or the color datainto multiple resolutions;

histogram generating means for generating a luminance histogram and/or acolor histogram which are histograms of the multi-resolution luminanceimage data and/or the multi-resolution color image data; and

quantification means for quantifying the sense of contrast based on theluminance histogram and/or the color histogram.

It is preferable for the first image processing apparatus of the presentinvention to further comprise processing means for carrying out imageprocessing on the image data based on the sense of contrast.

In this case, it is preferable for the processing means to carry out, asthe image processing, at least one of tone conversion processing,frequency enhancing processing, AE processing, and chroma conversionprocessing.

A second image processing apparatus of the present invention ischaracterized by the fact that image processing for changing luminanceinformation of an image represented by image data is carried out on theimage data based on color information of the image.

It is preferable for the second image processing apparatus of thepresent invention to comprise:

conversion means for obtaining color data representing the colorinformation from the image data;

unsharp image data generating means for generating unsharp image data ofthe color data;

histogram generating means for generating a histogram of the unsharpimage data; and

processing means for carrying out the image processing on the image databased on the histogram.

It is preferable for the histogram generating means to generate ahistogram representing a two-dimensional frequency distribution of theunsharp image data.

It is preferable for the second image processing apparatus of thepresent invention to comprise:

conversion means for obtaining color data representing color informationof the image from the image data;

multi-resolution conversion means for obtaining multi-resolution imagedata in a plurality of frequency bands by converting the color data intomultiple resolutions;

histogram generating means for generating a histogram ofmulti-resolution image data in a lowermost frequency band out of themulti-resolution image data in the plurality of frequency bands; and

processing means for carrying out the image processing on the image databased on the histogram.

The first image processing method and the second image processing methodof the present invention may be provided as a computer-readablerecording medium storing a program to cause a computer to execute themethods.

According to the present invention, the sense of contrast of an imagerepresented by image data can be quantified. Therefore, it is possibleto quantify not the contrast including various kinds of information ofan entire image, such as contrast found by a histogram of the imageitself, but a subjective sense perceived by a viewer of the image, suchas a shade difference of the entire image, a distribution of a lightportion and a dark portion in the image, a lightness distribution in asubject of interest, and information of colors included in the image.

Since not only information perceived by a viewer but also asubstantially large amount of information is included in an imagerepresented by image data, lightness information of the overall image isburied in a histogram generated from the image data. Meanwhile, bygenerating unsharp image data from the image data, an image representedby the unsharp image data does not include a detailed change in pixelvalues of a subject unlike the original image data, and represents theshade difference of the entire image perceived by a viewer. Therefore,the sense of contrast regarding an image can be quantified by beingbased on the unsharp image data or a histogram of luminance data and/orcolor data obtained from the image data.

The position information of a light portion and/or a dark portion in anunsharp image represents a position of a light subject and/or a darksubject in an image. Therefore, based on the position information, alightness distribution of the image can be represented as the sense ofcontrast.

In the case where histograms of multi-resolution image data in aplurality of frequency bands are generated by conversion of image datainto the multi-resolution image data, a histogram of the image data at aresolution in a lowermost frequency band represents a lightnessdistribution of the entire image, as does the histogram of unsharp imagedata. Meanwhile, a histogram of image data at a resolution in a mediumor high frequency band represents amplitude of a frequency componentcorresponding to the frequency band. For example, a shade of a facegenerated by the nose or hollows of the eyes, or a shade generated by abuilding or a subject is formed with the medium frequency componentwhile details of trees or flowers, a pattern and texture of a person'sclothes, a boundary (an edge) between objects and the like arerepresented by the high frequency component. Therefore, the more animage has local contrast, the wider a distribution width of thehistogram of the image data in the medium or high frequency bandbecomes. Consequently, the sense of contrast of the entire image datacan be quantified by the histogram of the image data in the lowfrequency band, and the sense of local contrast can also be quantifiedbased on the histogram of the image data in the medium and highfrequency bands. In this manner, not only the lightness distribution inan entire image but also a local lightness distribution can be found asthe sense of contrast.

Meanwhile, in the case where the luminance data and the color data areobtained from the image data and the luminance histogram and/or thecolor histogram of the multi-resolution image data are generated byconverting the luminance data and/or the color data into themulti-resolution image data, a luminance histogram in a low frequencyband represents a lightness distribution in an entire image while aluminance histogram in a medium to high frequency band represents anamplitude of a frequency component corresponding to the frequency band.Therefore, the sense of contrast of an entire image can be quantifiedbased on the luminance histogram in the low frequency band. Furthermore,the sense of local contrast can be represented based on the luminancehistogram in the medium and high frequency bands.

The color histogram in the low frequency band represents a chromadistribution of an image as a whole and the color histogram in themedium or high frequency band represents a chroma distributioncorresponding to the frequency band. Therefore, the sense of contrast ofan entire image based on colors therein can be quantified based on thecolor histogram in the low frequency band, and the sense of localcontrast can be quantified based on the color histogram in the mediumand high frequency bands.

Furthermore, by carrying out predetermined image processing on the imagedata based on the sense of contrast having been found, processed imagedata reflecting the sense of contrast perceived by a viewer can beobtained.

Moreover, by carrying out image processing for changing the luminanceinformation on the image data based on the color information of an imagerepresented by the image data, processed image data reflecting the senseof contrast perceived by a viewer and induced by the color of the imagecan be obtained.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing an outline configuration of an imageprocessing apparatus according to an embodiment of the presentinvention;

FIG. 2 is a block diagram showing an outline configuration of a firstembodiment of contrast-sense quantification means;

FIG. 3 shows illustrations explaining how unsharp image data aregenerated;

FIG. 4 is an example of a histogram of the unsharp image data;

FIG. 5 is another example of the histogram of the unsharp image data;

FIG. 6 is a diagram showing tone conversion look-up tables;

FIG. 7 is a flow chart showing processing carried out in the firstembodiment;

FIG. 8 is a histogram after converting the unsharp image data intohexadecimal numbers;

FIG. 9 is a block diagram showing an outline configuration of a secondembodiment of the contrast-sense quantification means;

FIGS. 10A and 10B show two-dimensional histograms;

FIG. 11 is a block diagram showing an outline configuration of a thirdembodiment of the contrast-sense quantification means;

FIG. 12 is a diagram explaining processing carried out in the thirdembodiment;

FIG. 13 is a diagram explaining processing carried out in a modificationof the third embodiment;

FIG. 14 is a block diagram showing an outline configuration of a fourthembodiment of the contrast-sense quantification means;

FIGS. 15A through 15C are illustrations of images represented by data atresolutions in each frequency band;

FIG. 16 shows histograms of the data at the resolutions in eachfrequency band;

FIG. 17 is a block diagram showing an outline configuration of a fifthembodiment of the contrast-sense quantification means;

FIG. 18 explains frequency enhancing processing;

FIG. 19 is a block diagram showing an outline configuration ofprocessing means carrying out AE processing;

FIGS. 20A and 20B explain the AE processing;

FIG. 21 is a block diagram showing an outline configuration of the imageprocessing apparatus according to another embodiment of the presentinvention;

FIG. 22 is a flow chart showing processing carried out in theembodiment; and

FIG. 23 is a block diagram showing an outline configuration of the imageprocessing apparatus according to still another embodiment of thepresent invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be explained withreference to the accompanying drawings.

FIG. 1 is a block diagram showing an outline configuration of an imageprocessing apparatus according to an embodiment of the presentinvention. The image processing apparatus according to this embodimentcomprises contrast-sense quantification means 1 for quantifying thesense of contrast C0 of an image represented by image data S0,processing means 2 for obtaining processed image data S1 by carrying outimage processing on the image data S0 based on the sense of contrast C0having been quantified by the contrast-sense quantification means 1, andoutput means 3 such as a printer or a CRT monitor for outputting theprocessed image data S1 as a visible image.

FIG. 2 is a block diagram showing a specific outline configuration ofthe contrast-sense quantification means 1. The contrast-sensequantification means 1 shown in FIG. 2 is explained as a firstembodiment. As shown in FIG. 2, the contrast-sense quantification means1 in the first embodiment comprises unsharp image generating means 11for generating unsharp image data Sus of the image data S0, histogramgenerating means 12 for generating a histogram Hus of the unsharp imagedata Sus, and quantification means 13 for finding the sense of contrastC0 of the image represented by the image data S0 based on the histogramHus.

In the contrast-sense quantification means 1 shown in FIG. 2, the senseof contrast C0 is quantified in the following manner. First, the unsharpimage data Sus of the image data S0 are generated by the unsharp imagegenerating means 11. The unsharp image data Sus are generated bycarrying out filtering processing using an unsharp mask filter on theimage data S0, for example. Examples of images represented by the imagedata S0 and the unsharp image data Sus are shown in FIG. 3. In FIG. 3,the images represented by the image data S0 and the unsharp image dataSus have the reference codes S0 and Sus respectively. In the unsharpimage data Sus, only a frequency band several percent of the Nyquistfrequency of the image data S0 exists. More specifically, the unsharpimage data Sus represent a frequency component of 0.5˜3 cycles/cm in theimage represented by the image data S0.

The histogram generating means 12 generates the histogram Hus of theunsharp image data Sus. FIG. 4 shows the histogram Hus of the unsharpimage data Sus together with a histogram H0 of the image data S0. InFIG. 4, pixel values are normalized to 0-100. As shown in FIG. 4, thehistogram H0 has a complex shape since the histogram includes not only alightness distribution in the entire image but also various kinds ofinformation in all subjects and every detail in the image. The shadedifference, that is, the contrast found from the histogram H0 rangesfrom 0-100, which is substantially wide. FIG. 5 shows histograms H0′ andHus′ generated from image data S0′ different from the image data S0 andunsharp image data Sus′ generated from the image data S0′. Comparison ofFIG. 4 with FIG. 5 reveals different shapes of the histograms. However,although the shapes of the two histograms H0 and H0′ are obviouslydifferent, the contrast found from the histogram H0′ in FIG. 5 rangeswidely from 0 to 100 as the contrast found from the histogram H0 shownin FIG. 4. Therefore, the contrast is not different between the images.

Meanwhile, the histogram Hus has a distribution width 27-92 narrowerthan that of the histogram H0, and excludes information of the detailsin the image. This distribution width represents the shade difference ofthe entire image, that is, the contrast of the image. Although thecontrast found by the histogram H0′ in FIG. 5 is the same as that of thehistogram shown in FIG. 4, a distribution width of the histogram Hus′ is17-75 which is different from the histogram Hus in FIG. 4. Therefore,the two histograms have different distribution widths and distributionpositions, and the contrast becomes different between the images. When aviewer observes an image, the viewer judges the contrast by observingthe entire image and not the details thereof. Therefore, in the firstembodiment, the distribution width of the unsharp image data Sus, thatis, a difference w between a maximum Husmax and a minimum Husmin in thehistogram Hus is found by the quantification means 13, and thedifference w is used as the sense of contrast C0 of the entire imageperceived by a viewer.

In the processing means 2, image processing is carried out on the imagedata S0 based on the sense of contrast C0 quantified by thecontrast-sense quantification means 1. A type of contrast of the imagerepresented by the image data S0 is found by comparing the sense ofcontrast C0, that is, the difference w between the maximum Husmax andthe minimum Husmin in the histogram Hus with a threshold value Th1 setin advance. The threshold value Th1 is set to approximately 50 in thecase where the pixel values of the entire histogram change in the range0-100 as shown in FIGS. 4 and 5. However, the threshold value is notalways limited to this example. If C0≧Th1, the image represented by theimage data S0 is judged as a high-contrast image, while the image isclassified as a low-contrast image if C0<Th1. In this case, twothreshold values Th1 and Th2 such as 0<Th2<Th1<100 may be set so thatthe image is judged to be a high-contrast image when C0>Th1 while astandard image and a low-contrast image in the case of Th2≦C0≦Th1 and inthe case of C0<Th2, respectively. Th1 is set to approximately 80 and Th2is set to around 40. However, the threshold values are not limited tothese numbers.

When the contrast type of the image is found in the above manner, one oftone conversion look-up tables (LUTs) prepared in accordance with thecontrast types is selected and tone conversion processing according tothe selected tone conversion LUT is carried out on the image data S0.FIG. 6 is a diagram showing the tone conversion LUTs. In thisembodiment, five tone conversion LUTs LUT1˜LUT5 are used. In the casewhere the image has been judged to be a high-contrast image, LUT5 isused. For the image judged to be a low-contrast image or a standardimage, the LUT1 or the LUT3 are used, respectively. The image processingfor obtaining the processed image data S1 is carried out by conversionof the tones of the image data S0 using the LUT described above. The LUTmay be selected from the LUT1 through LUT 5 according to the sense ofcontrast C0, that is, according to the difference w.

A tone curve represented by a function such as yout=a·yin+b(yout:output, yin:input) may be set by changing the parameters a and baccording to the contrast type or according to the sense of contrast C0(the value of w).

Operation of the first embodiment will be explained next.

FIG. 7 is a flow chart showing the operation of the first embodiment. Inthe flow chart shown in FIG. 7, the sense of contrast C0 is comparedwith the two threshold values Th1 and Th2. The unsharp image data Sus ofthe image data S0 are generated in the unsharp image generating means 11in the contrast-sense quantification means 1 (Step S1), and thehistogram Hus of the unsharp image data Sus is generated in thehistogram generating means 12 (Step S2). The quantification means 13finds the sense of contrast C0 based on the histogram Hus (Step S3). Thesense of contrast C0 having been found is then input to the processingmeans 2, and the processing means judges whether C0>th1 is confirmed ornot (Step S4). When the judgment result is affirmative, the image isjudged to be a high-contrast image and the tone conversion LUT5 isselected for the high-contrast image (Step S5). The tone conversionprocessing is carried out on the image data S0 (Step S6) to obtain theprocessed image data S1. The processed image data S1 are then output asa visible image from the output means 3 (Step S7).

When the judgment result at Step S4 is negative, the processing meansjudges whether C0<Th2 is confirmed or not (Step S8). When the judgmentresult is affirmative, the tone conversion LUT1 is selected (Step S9),since the image is judged to be a low-contrast image. Based on the LUT1,the tone conversion processing is carried out on the image data S0 (StepS6). When the judgment result is negative at Step S8, the image isjudged to be the standard image satisfying Th2≦C0≦Th1, and the toneconversion LUT3 is selected (Step 10). Based on the LUT3, the toneconversion processing is carried out on the image data S0 (Step S6).

In the image represented by the image data S0, not only informationperceived by the viewer of the image but also a large amount ofinformation is included. Therefore, lightness information of the entireimage is buried in the histogram H0 generated from the image data S0.Meanwhile, the image represented by the unsharp image data Sus of theimage data S0 represents the shade difference of the entire imageperceived by the viewer, since a detailed change in the pixel values ofa subject in the image is not included, unlike the image data S0.Therefore, the sense of contrast C0 perceived by the viewer can bequantified by being based on the histogram Hus of the unsharp image dataSus. By carrying out the tone conversion processing on the image data S0based on the sense of contrast C0, the processed image data S1representing an image reflecting the sense of contrast C0 perceived bythe viewer can be obtained.

In the first embodiment, the histogram H0 is generated from the unsharpimage data Sus. However, in the case where the unsharp image data Sushave 8-bit data values (0-255), the unsharp image data may be convertedinto hexadecimal numbers and the histogram may be generated from theunsharp image data Sus having the hexadecimal numbers. At this time,since the unsharp image data Sus, unlike the image data S0, do notinclude information of the detail of the image, the distribution widthof the histogram Hus generated after the conversion into hexadecimalnumbers does not change substantially as shown in FIG. 8 when comparedwith the histogram generated from the 8-bit unsharp image data Sus.Consequently, the sense of contrast C0 obtained therefrom does notchange greatly before and after the conversion into hexadecimal numbers.Since the amount of pixel-value data is reduced if the conversion intohexadecimal numbers is carried out, the histogram can be generatedeasily. Therefore, by generating the histogram after the conversion ofthe unsharp image data Sus into hexadecimal numbers is carried out,processing can be made faster. In this case, the data values of theunsharp image data Sus have been converted into hexadecimal numbers.However, if the data values are to be made smaller than the 8-bit data,the histogram may be generated after conversion into octal numbers or32-bit numbers, for example.

In the above embodiment, the sense of contrast C0 has been defined asthe difference w between the maximum Husmax and the minimum Husmin inthe histogram Hus. However, a pixel value at a position smaller than themaximum Husmax by 10% of (Husmax-Husmin) and a pixel value at a positionlarger than the minimum Husmin by 10% of (Husmax-Husmin) may be found sothat the sense of contrast C0 is defined as a difference between thesepixel values.

Furthermore, in the first embodiment described above, the unsharp imagedata Sus have been generated from the image data S0 to quantify thesense of contrast C0 based on the histogram Hus of the unsharp imagedata Sus. However, the image data S0 may be converted into luminancedata and color data representing luminance information and colorinformation of the image represented by the image data S0 so that thesense of contrast C0 can be quantified based on a histogram of theluminance data and/or the color data. Hereinafter, this method will beexplained as a second embodiment.

FIG. 9 is a block diagram showing an outline configuration of the secondembodiment of the contrast-sense quantification means 1. As shown inFIG. 9, the contrast-sense quantification means 1 in the secondembodiment further comprises conversion means 15 for converting theimage data S0 into luminance data L* and color data C*, in addition tothe unsharp image generating means 11, the histogram generating means12, and the quantification means 13 in the first embodiment. Unsharpluminance image data Lus or unsharp color image data Cus which areunsharp image data of the luminance data L* or the color data C* aregenerated by the unsharp image generating means 11, and the histogramgenerating means 12 generates a luminance histogram HLus which is ahistogram of the unsharp luminance image data Lus and a color histogramHCus which is a histogram of the unsharp color image data Cus.

The conversion means 15 converts the image data S0 into the luminancedata L* and the color data C* in the following manner. In the secondembodiment, the image data S0 are assumed to comprise RGB color data R0,G0, and B0 following the standard of ITU-R BT.709(REC.709). In theconversion means 15, the color data R0, G0, and B0 comprising the imagedata S0 are converted into CIE1931 tristimulus values X, Y, and Z basedon the following Equations (1) to (3):Pr=R0/255Pg=G0/255  (1)Pb=B0/255R1′=((Pr+0.099)/1.099)^(2.222)G1′=((Pg+0.099)/1.099)^(2.222) (if Pr,Pg,Pb≧0.081)  (2)B1′=((Pb+0.099)/1.099)^(2.222)R1′=Pr/4.5G1′=Pg/4.5 (if Pr,Pg,Pb<0.081)  (2′)B1′=Pb/4.5X R0′Y=|A|·G0′  (3)Z B0′where the matrix |A| is a matrix for converting the color data R0′, G0′,and B0′ into the tristimulus values X, Y, and Z, and the followingnumbers can be used, for example:0.4124 0.3576 0.1805|A|=0.2126 0.7152 0.0722  (4)0.0193 0.1192 1.0571

The tristimulus values X, Y, and Z may be found by using a look-up tableinstead of the matrix |A|.

CIE1976 L*, a*, and b* are found from the tristimulus values X, Y, and Zaccording to the following Equations (5) to (7):a*=500{f(X/Xn)−f(Y/Yn)}  (5)b*=200{f(Y/Yn)−f(Z/Zn)}  (6)L*=116 (Y/Yn)^(1/3)−16 (if Y/Yn>0.008856)  (7)L*=903.25(Y/Yn) (if Y/Yn≦0.008856)When X/Xn, Y/Yn, Z/Zn>0.008856,f(a/an)=(a/an)^(1/3) (a=X, Y, Z)When X/Xn, Y/Yn, Z/Zn≦0.008856,f(a/an)=7.787(a/an)+16/116The values Xn, Yn, and Zn are tristimulus values for a white color andcorresponding to CIE-D65 (a light source whose color temperature is 6500K). The chroma C* is then found according to the following Equation (8):C*=(a* ² +b* ²)^(1/2)  (8)L* and C* are then output as the luminance data L* and the color dataC*.

From the luminance data L* or the color data C*, the unsharp luminanceimage data Lus and the unsharp color image data Cus which are theunsharp image data of the luminance data L* or the color data C* aregenerated by the unsharp image generating means 11, as in the firstembodiment. For the color data C*, a degree of unsharpness can be lowerthan that of the luminance data L*, taking not only an overall colorchange but also an influence of the medium frequency component such asfine glasses and flowers into consideration. More specifically, theunsharp color image data Cus preferably represents frequency componentssuch as 0.5˜10 cycles/cm in the image represented by the image data S0.

As in the first embodiment, the histogram generating means 12 generatesthe luminance histogram HLus or the color histogram HCus from theunsharp luminance image data Lus or the unsharp color image data Cus,respectively.

The quantification means 13 finds the distribution width of theluminance histogram HLus or the color histogram HCus to be output as thesense of contrast Co.

Based on the sense of contrast having been found in the above manner,the processing means 2 carries out image processing on the image dataS0. More specifically, the type of contrast of the image represented bythe image data S0 is found based on the sense of contrast C0, as in thefirst embodiment. The tone conversion LUT according to the type isselected and the tone conversion processing is carried out on the imagedata S0 by using the tone conversion LUT having been selected.

In the case where the sense of contrast C0 is found from the color dataC* in the second embodiment, chroma conversion processing for causingthe chroma of the image represented by the image data S0 to increase maybe carried out if the type of contrast found from the sense of contrastC0 is low. More specifically, the chroma is improved by multiplying thechroma C* found from Equation (8) with an enhancement coefficient αc. Asa value of the coefficient αc, approximately 1.2 is preferable. However,the value is not limited to this example. Furthermore, the toneconversion processing and the chroma conversion processing may becarried out at the same time.

In the case where the chroma is improved, not only multiplication withthe uniform coefficient αc for the entire image represented by the imagedata S0 but also multiplication with αc as a function of chroma may becarried out so that an area of low chroma in the image has more improvedchroma. The coefficient αc may be changed in accordance with a hue angleH (=tan⁻¹(b*/a*)).

In the second embodiment, the conversion means 15 finds the chroma C*according to Equation (8) and uses C* as the color data. However, a* andb* found from Equations (5) and (7) may be used as the color data. Inthis case, the unsharp image generating means 11 finds unsharp colorimage data aus and bus of a* and b*, and the histogram generating means12 generates a two-dimensional histogram Hab from the unsharp colorimage data aus and bus. FIG. 10 shows an example of the two-dimensionalhistogram Hab. In FIG. 10, a distance from the origin represents thechroma. The sharper a color is, the farther the color is located fromthe origin. Therefore, when the image represented by the image data S0has a multitude of vivid colors, a distribution of the two-dimensionalhistogram Hab is widened. For example, in FIGS. 10( a) and 10(b), theimage from which the two-dimensional histogram Hab shown in FIG. 10( b)is generated has sharper colors, since the distribution is wider in FIG.10( b).

Therefore, by finding a distribution area Ac of the two-dimensionalhistogram Hab, the area can be used as the sense of contrast C0. Basedon the sense of contrast C0, the type of contrast of the imagerepresented by the image data S0 is found as in the first embodiment,and image processing such as the tone conversion processing and thechroma conversion processing can be carried out on the image data S0.

In the second embodiment, the sense of contrast C0 may be defined as aratio R=P/P_(all) where P is the number of pixels having the color dataC* equal to or larger than a predetermined threshold value and P_(all)is the number of all the pixels in the image represented by the imagedata S0. More specifically, in the aus-bus plane shown in FIG. 10, acircular area centering on the origin and having a predetermined radius(a radius corresponding to the threshold value) is set and pixels notincluded in the circle are counted as the pixels P and the ratio R whichis the ratio of P to the number of all pixels P_(all) is found as thesense of contrast C0.

In the first embodiment described above, the histogram Hus of theunsharp image data Sus of the image data S0 have been generated, and thesense of contrast C0 has been quantified based on the histogram Hus.However, the sense of contrast C0 may be found as a distribution of alight portion and/or a dark portion in an image. Hereinafter, thismethod will be explained as a third embodiment. FIG. 11 is a blockdiagram showing an outline configuration of the third embodiment of thecontrast-sense quantification means 1. As shown in FIG. 11, thecontrast-sense quantification means 1 in the third embodiment comprisesunsharp image generating means 21 for generating the unsharp image dataSus of the image data S0, hexadecimal conversion means 22 for obtaininghexadecimal unsharp image data Sus16 by converting the unsharp imagedata Sus into hexadecimal numbers 0-15, position detection means 23 fordetecting a position of a pixel having the maximum number 15 in an imagerepresented by the hexadecimal unsharp image data Sus16, and operationmeans 24 for finding a distance between the pixel position detected bythe position detection means 23 and the center O of the image and forcalculating a standard deviation σ of the distance defined as the senseof contrast C0. The sense of contrast C0 calculated in the above mannerquantifies a distribution of the light portion perceived by a viewer ofthe image.

When the standard deviation σ calculated by the operation means 24 iscomparatively small, the image is judged to be an image having the lightportion concentrated around the center thereof (for example, a flashimage obtained by photographing using a flash), and image processingadequate therefor is carried out by the processing means 2. In the thirdembodiment, the processing means 2 judges whether or not the sense ofcontrast C0, that is, the standard deviation σ found by thecontrast-sense quantification means 1 is smaller than a predeterminedthreshold value Th5. When σ<Th5, the image is judged to be a flash imagehaving the light portion concentrated around the center thereof, andimage processing adequate therefor is carried out on the image data S0.When σ≧Th5, the image is judged to be a standard image and imageprocessing adequate therefor is carried out on the image data S0.

In a flash image, the contrast increases, since strong light isirradiated on a subject thereof. Therefore, the subject appears whiterreflecting the flash. For this reason, the processing means 2 extractsthe light portion from the image represented by the image data S0 whenσ<Th5, and carries out the tone conversion processing using the LUT5shown in FIG. 6 on the image data S0 in the portion so as to suppressthe contrast. In this manner, the contrast in the light portion issuppressed and it is possible to obtain processed image data S1representing an image having no strong reflection.

In the third embodiment described above, the sense of contrast C0 hasbeen defined as the standard deviation σ of the distance between thepixel position found by the position detection means 23 and the imagecenter O. However, as shown in FIG. 13, an area A1 having apredetermined area around the center O is set and the number of pixelshaving the hexadecimal number 15 in the area A1 may be counted as thesense of contrast C0. In this case, the processing means 2 judgeswhether or not the sense of contrast C0, that is, the number of thepixels having the hexadecimal number 15 in the area A1, is larger than apredetermined threshold value Th6. When C0>Th6, the image is judged tobe an image having a light portion concentrated around the centerthereof, and the processing means 2 carries out the tone conversionprocessing for suppressing the contrast in the area A1. When C0≦Th6, theimage is judged to be a standard image and the processing means 2carries out the image processing adequate therefor. In this manner, theprocessed image data S1 representing the image having no strongreflection and suppressed contrast in the light portion can be obtained.

In the first embodiment, the unsharp image data Sus of the image data S0have been generated and the histogram of the unsharp image data Sus hasbeen found to be used for quantifying the sense of contrast C0. However,the image data S0 may be converted into a multi-resolution space in aplurality of frequency bands so that histograms of the data at theresolutions in the frequency bands are generated to be used forquantification of the sense of contrast C0. Hereinafter, this methodwill be explained as a fourth embodiment.

FIG. 14 is a block diagram showing an outline configuration of thecontrast-sense quantification means 1 in the fourth embodiment. As shownin FIG. 14, the contrast-sense quantification means 1 in the fourthembodiment comprises multi-resolution conversion means 31 for obtainingmulti-resolution image data (hereinafter called resolution data) RL, RM,and RH in a low frequency band, a medium frequency band, and a highfrequency band respectively by converting the image data S0 into amulti-resolution space by using a method such as wavelet transform andLaplacian pyramid, area extracting means 32 for extracting, as a lightportion M1, an area in which pixel values are equal to or larger than apredetermined threshold value Th7 from the resolution data RL in the lowfrequency band, histogram generating means 33 for generating histogramsHM and HH of an area corresponding to the light portion M1 regarding theresolution data RM and RH in the medium frequency band and in the highfrequency band, and quantification means 34 for quantifying the sense ofcontrast C0 of the image represented by the image data S0.

In the contrast-sense quantification means 1 in the fourth embodiment,the sense of contrast C0 is found in the following manner. First, theimage data S0 are converted into the multi-resolution space by themulti-resolution conversion means 31 and the resolution data RL, RM andRH in the low, medium, and high frequency bands are generated. Theresolution data RL in the low frequency band include information oflightness while the resolution data RM and RH in the medium and highfrequency bands represent frequency components only. In FIG. 15, FIGS.15( a), 15(b), and 15(c) show images represented by the resolution datain the low frequency band, in the medium frequency band, and in the highfrequency band, respectively.

The resolution data RL in the low frequency band are input to the areaextracting means 32 and the area in which the pixel values are equal toor larger than the threshold value Th7 is extracted as the light portionM1 to be input to the histogram generating means 33. Meanwhile, thenumber n of the pixels in the light portion M1 is input to thequantification means 34. The histogram generating means 33 generates thehistograms HM and HH corresponding to the light portion M1 regarding theresolution data RM and RH in the medium and high frequency bands.

A distribution width BL of a histogram HL of the resolution data RL inthe low frequency band shows a distribution of pixel values as in FIG.16( a), and represents overall lightness of the image, as do thehistograms shown in FIGS. 4 and 5 do. Distribution widths BM and BH ofthe histograms HM and HH of the resolution data RM and RH in the mediumand high frequency bands show amplitudes of frequencies centering on 0as shown in FIGS. 16( b) and (c).

For example, a shade of a face generated by the nose or hollows of theeyes, and a shade generated by a building or a subject is formed withfrequency components in the medium frequency band higher than the lowfrequency band. Therefore, the more an image has local contrast of asubject such as a face, the larger the local shade becomes. As a result,the amplitude of the resolution data RM in the medium frequency bandbecomes large. Meanwhile, details of trees or flowers, a pattern andtexture of a person's clothes, a boundary (an edge) between objects andthe like are formed with the high frequency component. Therefore, thelarger the contrast of a local area corresponding to the detailedstructure becomes, the clearer the detailed structure appears. As aresult, the amplitude of the resolution data RH in the high frequencyband becomes larger.

The quantification means 34 quantifies the sense of contrast C0 based onthe histograms HM and HH. First, the distribution width BM in thehistogram HM of the resolution data RM in the medium frequency band iscompared with a predetermined threshold value Th8. If the distributionwidth BM of the histogram HM is larger than the threshold value Th8(BM>Th8), the image is judged to be a standard image having acomparatively large amount of information of the medium frequency band.If the width BM is equal to or smaller than the threshold value Th8(BM≦Th8), the image is judged to be a low-contrast image not including asubstantially large amount of information of the medium frequency band.In the case where the image is judged to be the standard image, thedistribution width BH in the histogram HH of the resolution data RH inthe high frequency band may be compared with a predetermined thresholdvalue Th9 so that the image can be judged to be a high-contrast imageincluding a comparatively large amount of high frequency information ifthe distribution width BH is larger than the threshold value Th9(BH>Th9). Otherwise (BH≦Th9) the image is judged to be the standardimage not including a substantial amount of the high frequencyinformation. Meanwhile, the number n of the pixels in the light portionM1 extracted by the area extracting means 32 may be compared with apredetermined threshold value Th10 so that the image is judged to be thelow-contrast image if the number n is smaller than the threshold valueTh10 (n<Th10). In this case, if the number n is equal to or larger thanthe threshold value Th10, the type of contrast is judged by using theresolution data RM and RH in the medium and high frequency bands, as hasbeen described above.

After the type of contrast has been found in the above manner, the typeof contrast is output as the sense of contrast C0. In this case, thesense of contrast C0 is a signal having a number corresponding to thetype of contrast, such as 1 for. the low-contrast image, 2 for thehigh-contrast image, and 3 for the standard image, for example.

The processing means 2 carries out on the image data S0 the imageprocessing for converting the tone by selecting the tone conversion LUTas in the first embodiment, based on the sense of contrast C0 quantifiedby the contrast-sense quantification means 1. In this manner, theprocessed image data S1 are obtained.

In the fourth embodiment described above, the luminance data L* or thecolor data C* of the image data S0 may be found as in the secondembodiment. Resolution data of the luminance data L* or the color dataC* are then obtained by converting the luminance data L* or the colordata C* into the multi-resolution space. The sense of contrast C0 isthen quantified based on the resolution data. Hereinafter, this methodwill be explained as a fifth embodiment.

FIG. 17 is a block diagram showing an outline configuration of thecontrast-sense quantification means 1 in the fifth embodiment. As shownin FIG. 17, the contrast-sense quantification means 1 in the fifthembodiment comprises the conversion means 15 in the second embodiment,in addition to the multi-resolution conversion means 31, the areaextracting means 32, the histogram generating means 33 and thequantification means 34 in the fourth embodiment. The multi-resolutionconversion means 31 converts the luminance data L* or the color data C*into the multi-resolution space and obtains luminance resolution dataRLL, RML, and RHL or color resolution data RLC, RMC, and RHC in the low,medium, and high frequency bands.

In the case where only the luminance resolution data RLL, RML and RHLare obtained, the area extracting means 32 extracts the light portion M1from the luminance resolution data RLL in the low frequency band as inthe fourth embodiment, and the histogram generating means 33 generatesluminance histograms HML and HHL corresponding to the light portion M1regarding the resolution data RML and RHL in the medium and highfrequency bands, respectively. The quantification means 34 thenquantifies the sense of contrast C0 based on the luminance histogramsHML and HHL.

In the case where only the color resolution data RLC, RMC and RHC areobtained, the area extracting means 32 extracts a high-chroma area M2having data values equal to or larger than a predetermined thresholdvalue from an image represented by the color resolution data RLC in thelow frequency band.

The color resolution data RMC in the medium frequency band represents ashade of a face generated by the nose or hollows of the eyes and a shadegenerated by a building or a subject, as the resolution data RM of theimage data S0 in the medium frequency band. The color resolution dataRHC in the high frequency band also represents detailed structures suchas details of trees or flowers, a pattern and texture of a person'sclothes, and a boundary (an edge) between objects, as the resolutiondata RH of the image data S0 in the high frequency band.

Therefore, the histogram generating means 33 generates color histogramsHMC and HHC of the color resolution data RMC and RHC in the medium andhigh frequency bands regarding the high-chroma area M2 extracted by thearea extracting means 32. As in the fourth embodiment above, thequantification means 34 then judges the type of contrast by comparingthe amplitudes of the color histograms HMC and HHC with a predeterminedthreshold value, and the judgment result can be obtained as the sense ofcontrast C0.

In the case where the sense of contrast C0 is found from the color dataC*, the chroma conversion processing for enhancing the chroma of theimage represented by the image data S0 may be carried out based on thesense of contrast C0 if the type of contrast is a low-contrast image.More specifically, the chroma is improved by multiplication of thechroma C* found by using Equation (8) with the enhancement coefficientαc. As the value of αc, approximately 1.2 is preferable, but it is notlimited to this value.

In the fifth embodiment described above, if both the luminanceresolution data RLL, RML and RHL and the color resolution data RLC, RMC,and RHC are obtained, the light portion M1 is extracted based on theluminance data RLL in the low frequency band and the color histogramsHMC and HHC corresponding to the light portion M1 are generated for thecolor resolution data RMC and RHC in the medium and high frequencybands. Based on the color histograms HMC and HHC, the sense of contrastC0 can be quantified. Alternatively, the high-chroma area M2 may beextracted based on the color resolution data RLC in the low frequencyband so that the luminance histograms HML and HHL corresponding to thehigh-chroma area M2 are generated for the luminance resolution data RMLand RHL in the medium and high frequency bands to quantify the sense ofcontrast C0.

In the first and third embodiments described above, the sense ofcontrast C0 has been found as the distribution width of the histogramsof the image data S0, the luminance data L* and the color data C*, whilethe standard deviation of the image data S0 has been found as the senseof contrast C0 in the second embodiment. However, regarding the imagedata S0, various kinds of information may be found as characteristicquantities making it possible to find the distribution widths andstandard deviations of the histograms of the image data S0, theluminance data L* and the color data C* as well as the sense of contrastC0 of the image data S0. The sense of contrast C0 is then quantified byweighting and adding the characteristic quantities according to Equation(9) below:

$\begin{matrix}{{C\; 0} = {\sum\limits_{n}{{an} \cdot {vn}}}} & (9)\end{matrix}$where vn is the characteristic quantity, an is a weight, and n is thenumber of the characteristic quantities. The weight an may be foundempirically. In other words, the sense of contrast C0 is found whilechanging the weight, and visual evaluation is carried out on a pluralityof images generated through different image processing in accordancewith the sense of contrast C0. By selecting an image agreeing with thesense of contrast perceived visually, the weight an used at the time ofgenerating the image is used for Equation (9).

In the first to third embodiments described above, the sense of contrastC0 has been quantified. However, the type of contrast of the image maybe found as the sense of contrast, as in the fourth and fifthembodiments.

In the first to third embodiments, the type of contrast may be judged inaccordance with a result of weighted addition of the characteristicquantities vn as shown in Equation (10) below, and the judgment resultmay be used as the sense of contrast C0:

$\begin{matrix}{{{Ph} = {\sum\limits_{n}{{{hn} \cdot {vn}}\mspace{11mu}\left( {{for}\mspace{14mu}{high}\text{-}{contrast}\mspace{14mu}{image}} \right)}}}{{Ps} = {\sum\limits_{n}{{{sn} \cdot {vn}}\mspace{11mu}\left( {{for}\mspace{14mu}{standard}\text{-}{contrast}\mspace{14mu}{image}} \right)}}}{{Pl} = {\sum\limits_{n}{{\ln \cdot {vn}}\mspace{11mu}\left( {{for}\mspace{14mu}{low}\text{-}{contrast}\mspace{14mu}{image}} \right)}}}} & (10)\end{matrix}$Here, hn, sn, and ln are weights for calculating Ph,Ps, and Pl asindices representing probabilities of an image being a low-contrastimage, a standard image and a high-contrast image, respectively. Forexample, if Ph, Ps, and Pl are respectively calculated to be 10%, 30%,and 70% for an image and 80%, 40%, and 5% for another image, the formeris judged to be a low-contrast image and the latter is judged to be ahigh-contrast image. In this case, the sense of contrast C0 is a signalhaving a number in accordance with the type of contrast, such as 1 forthe low-contrast image, 2 for the high-contrast image and 3 for thestandard image. Based on the type of contrast represented by the senseof contrast C0, the tone conversion LUT is selected and the toneconversion processing is carried out on the image data S0.

In the fourth embodiment above, the sense of contrast C0 has beenquantified based on the number n of the pixels in the light portion M1obtained from the resolution data RL in the low frequency band and fromthe distribution width BM and BH of the histograms HM and HH of theresolution data RM and RH in the medium and high frequency bands.However, the number of pixels in the light portion, the distributionwidth of the histogram, or the standard deviation used in the thirdembodiment may be found as the characteristic quantities for theresolution data RL, RM, and RH in the low, medium, and high frequencybands so that the sense of contrast C0 in each frequency band can bequantified by weighted addition of the characteristic quantitiesaccording to Equation (9). Furthermore, as shown by Equation (11) below,by weighted addition of the sense of contrast C0 found for each of thefrequency bands, the sense of contrast may be quantified. Thecharacteristic quantities may include the number n of the pixels in thelight portion M1 obtained from the luminance resolution data RLL in thelow frequency band, the distribution widths of the luminance histogramsRML and RHL in the medium and high frequency bands, and/or thedistribution widths of the color histograms HMC and HHC of the colorresolution data RMC and RHC in the medium and high frequency bands inthe high-chroma area M2 obtained from the color resolution data RLC inthe low frequency band used in the fifth embodiment:

$\begin{matrix}{{C\; 0} = {{L \cdot {\sum\limits_{i}{\alpha\;{i \cdot {li}}}}} + {M \cdot {\sum\limits_{j}{\beta\;{j \cdot {mj}}}}} + {H \cdot {\sum\limits_{k}{\gamma\;{k \cdot {hk}}}}}}} & (11)\end{matrix}$where li, mj, hk are the characteristic quantities in each of thefrequency bands, αi, βj, and γk are weights corresponding to an inEquation (9), L, M, and H are weights in each of the frequency bands,and i, j, and k are the numbers of the characteristic quantities in eachof the frequency bands. The weights αi, βj, and γk and L, M, and K arefound empirically as the weight an in Equation (9) above. In Equation(11), if L=1, M=0, and H=0, Equation (11) is practically equivalent toEquation (9).

In this case, the image may be classified such as the low-contrastimage, the standard image, and the high-contrast image, according to thevalue of the sense of contrast C0 found by using Equation (9) or (11),and the classification result may be used as the sense of contrast C0.In this case, the sense of contrast C0 is a value representing thelow-contrast image, the standard image or the high-contrast image.

In the fourth and fifth embodiments described above, the probability ofthe type of contrast may be found as in Equation (10) so that the typeof contrast of an image can be found based on the probability. Thejudgment result is then used as the sense of contrast C0.

In the above embodiments, the processing means 2 carries out the toneconversion processing by changing the tone conversion LUT according tothe sense of contrast C0 and/or performing chroma conversion processingon the image data S0. However, the image processing is not limited tothe processing above. For example, the sense of contrast C0 is comparedwith a predetermined threshold value Th11 and if C0<Th11, frequencyprocessing for enhancing the sense of contrast, such as enhancement of afrequency component shown by a hatched portion in FIG. 18 by usingEquation (12) below may be carried out on the image data S0. In FIG. 18,an Fx axis and an Fy axis represent frequencies in a Fourier Plane andthe hatched portion corresponds to a high frequency component in theimage data S0:F′(x, y)=F(x,y)+β{F(x,y)−F(x,y)·d(C0)}  (12)where F(x,y) represents image data S0, F′(x,y) is the processed imagedata S1, d(C0) is a function for determining a degree of unsharpness,and β is an enhancement coefficient.

Furthermore, in the case where the image data S0 represents an imageincluding a human face, a face area corresponding to the face may beextracted from the image data S0 so that AE processing (automaticexposure control processing) for changing the brightness of the facearea is carried out according to the density of the face area and thesense of contrast C0 having been quantified. In this manner, theprocessed image data S1 may be obtained. Hereinafter, processing meansfor carrying out the AE processing will be explained. FIG. 19 is a blockdiagram showing an outline configuration of the processing means 2 forcarrying out the AE processing. As shown in FIG. 19, the processingmeans 2 comprises face extracting means 41 for extracting an area of ahuman face from the image represented by the image data S0, densitycalculating means 42 for finding a density tface of the face areaextracted by the face extracting means 41, and AE processing means 43for obtaining the processed image data S1 by carrying out the AEprocessing on the image data S0 based on the sense of contrast C0 andthe density tface of the face area.

As a method of extracting the face by using the face extracting means41, the image may be divided based on a distribution of hue and chromaof the image represented by the image data S0 so that a face candidatearea is extracted, and the face area is extracted from a shape of anarea neighboring the face candidate area, as described in JapaneseUnexamined Patent Publication No. 6(1994)-67320, for example.Alternatively, an ellipse circumscribing a simply-extracted facecandidate area is found and the area surrounded by the ellipse may beused as the face area. Furthermore, the face area may be extracted byusing a neural network described in Japanese Unexamined PatentPublication Nos. 5(1993)-274438 and 5(1993)-307605, for example.

The density calculating means 42 calculates an average of the pixelvalues in the face area extracted by the face extracting means 41 as theface area density tface.

When the AE processing is carried out, the sense of contrast C0 input tothe processing means 2 represents the distribution width of thehistogram Hus quantified as in the first embodiment.

The face extracting means 41 extracts the face area from the imagerepresented by the image data S0 and the density calculating means 42calculates the density tface of the face area. The AE processing means43 carries out the AE processing on the image data S0 based on the senseof contrast C0 and the density tface of the face area. In the AEprocessing means, after the AE processing is carried out on the entireimage data S0, further AE processing is then carried out only on theface area in order to change the density of the face adequately. FIG. 20is a diagram explaining the AE processing. In the AE processing means43, the sense of contrast C0 is compared with predetermined thresholdvalues Th12 and Th14 (Th12<Th14). If C0<Th12, the image is judged to bea low-contrast image and the density tface of the face area is comparedwith a predetermined threshold value Th13. If tface<Th13, the face isjudged to be too dark and the AE processing is carried out on the imagedata S0 so that the density tface of the face area agrees with Th13 asshown in FIG. 20( a). In this manner, the processed image data S1 areobtained.

If C0<Th12 and tface≧Th13, the brightness of the face area is judged tobe adequate and the AE processing is not carried out.

If C0≧Th12 and tface<Th13, the sense of contrast C0 is judged to beadequate but the face area is judged to be too dark. Therefore, theprocessed image data S1 are obtained by carrying out the AE processingon the image data S0 in order to cause the density tface of the facearea to become equal to (Th13+tface)/2.

If C0≧Th12 and tface≧Th13, the brightness of the face area is judged tobe adequate and no AE processing is carried out.

Meanwhile, if C0>Th14, the image is judged to be a high-contrast imageand the density tface of the face area is compared with a predeterminedthreshold value Th15. If tface>Th15, the face area is judged to be toobright and the AE processing for causing the density tface of the facearea to agree with the threshold value Th15 is carried out, as shown inFIG. 20( b). In this manner, the processed image data S1 are obtained.In this case, it is preferable for the AE processing to be carried outonly on the face area.

If C0>Th14 and tface≦Th15, the brightness of the face area is judged tobe adequate and the AE processing is not carried out.

If C0≦Th14 and tface>Th15, the sense of contrast C0 is judged to beadequate but the face area is judged to be too bright. Therefore, the AEprocessing for causing the density tface of the face area to becomeequal to (Th15+tface)/2 is carried out on the image data S0. In thismanner, the processed image data S1 are obtained.

If C0≦Th14 and tface≦Th15, the brightness of the face area is judged tobe adequate and no AE processing is carried out.

In the third embodiment described above, if σ<Th15, the image is judgedto be a flash image having a face area reflecting the light, and the AEprocessing for causing the density tface of the face area to becomesmaller may be carried out.

In the embodiments described above, the processing means 2 carries outthe tone conversion processing, the chroma conversion processing,frequency enhancing processing or the AE processing. However, thesekinds of processing may be used in combination, such as the toneconversion processing and the AE processing. Furthermore, various kindsof processing other than those described above may be used.

Another embodiment of the present invention will be explained next. FIG.21 is a block diagram showing an outline configuration of an imageprocessing apparatus according to the embodiment. As shown in FIG. 21,the image processing apparatus comprises conversion means 51 forconverting the image data S0 into the luminance data L* and the colordata C* as the conversion means 15 in the second embodiment,multi-resolution conversion means 52 for obtaining the luminanceresolution data RLL, RML, and RHL and the color resolution data RLC, RMCand RHC in the low, medium and high frequency bands by carrying outmulti-resolution conversion on the luminance data L* and on the colordata C*, histogram generating means 53 for generating the luminancehistogram HLL of the luminance resolution data RLL in the low frequencyband and the color histogram HLC of the color resolution data RLC in thelow frequency band, pattern setting means 54 for setting a pattern J forimage processing to be carried out on the image data S0 based on theluminance histogram HLL and the color histogram HLC, processing means 55for obtaining the processed image data S1 by carrying out imageprocessing on the image data S0 according to the pattern J having beenset, and output means 56 for outputting the processed image data S1 as avisible image.

In the case where the color data C* can be obtained as chroma, the colorhistogram HLC in one dimension is generated. Meanwhile, the colorhistogram HLC in two dimensions as shown in FIG. 10 is generated in thecase where the color data are obtained as a* and b*.

In the processing pattern setting means 54, the tone conversion LUTshown in FIG. 6 and a value of the enhancement coefficient αc forimproving the chroma C* are set as the processing pattern J. First,characteristic quantities representing the distributions of theluminance histogram HLL and the color histogram HLC are found from thehistograms. The distribution width of the luminance histogram HLL isused as a characteristic quantity P1. If the color histogram HLC isone-dimensional, the distribution width thereof is used as acharacteristic quantity P2 while a distribution area of the colorhistogram HLC is used as P2 if the color histogram HLC istwo-dimensional. The characteristic quantity P2 is then compared with apredetermined threshold value Thl6. If P2≧Th16, the tone conversion LUT3is selected for the tone conversion from the tone conversion LUTs shownin FIG. 6, and the coefficient αc is set to 1.0.

Meanwhile, if P2<Th16, the characteristic quantity P1 found from theluminance histogram HLL is compared with a predetermined threshold valueTh17. If P1≧Th17, the image is judged to be a high-contrast image. TheLUT5 is selected and αc is set to 1.0. If P1<Th17, the image is judgedto be a low-contrast image, and the LUT1 and αc=1.2 are set.

The image processing is carried out by the processing means 55 accordingto the processing pattern J set in the above manner.

An operation of this embodiment will be explained next. FIG. 22 is aflow chart showing the operation of this embodiment. In the conversionmeans 51, the image data S0 are converted into the luminance data L* andthe color data C* (Step S11). In the multi-resolution conversion means52, the luminance data L* and the color data C* are converted intomultiple resolutions and the luminance resolution data RLL, RML, and RHLand the color resolution data RLC, RMC, and RHC in the low, medium, andhigh frequency bands are obtained (Step S12). In the histogramgenerating means 53, the luminance histogram HLL and the color histogramHLC are generated from the luminance resolution data RLL and the colorresolution data RLC in the low frequency band (Step S13).

The characteristic quantities P1 and P2 of the luminance histogram HLLand the color histogram HLC are respectively calculated (Step S14), andwhether or not P2≧Th16 is satisfied is judged (Step S15). If thejudgment result at Step S15 is confirmative, the tone conversion LUT3 isselected and the enhancement coefficient αc is set to 1.0 (Step S16) toset the processing pattern J. If the judgment result at Step S15 isnegative, whether or not P1≧Th17 is judged (Step S17). If the judgmentresult at Step S17 is affirmative, the LUT5 is selected and theenhancement coefficient αc is set to 1.0 (Step S18) to set theprocessing pattern J. If the judgment result at Step S17 is negative,the LUT1 is selected and the coefficient αc is set to 1.2 (Step S19). Inthis manner, the processing pattern J is set and the image processing iscarried out on the image data S0 according to the processing pattern J(Step S20) to obtain the processed image data S1. The processed imagedata S1 obtained in the above manner are output as a visible image bythe output means 56 (Step S21).

In the above embodiment, the luminance data L* and the color data C* aresubjected to multi-resolution conversion and the histogram HLL of theresolution data RLL in the low frequency band and the histogram HLC ofthe color resolution data RLC in the low frequency band are generated.However, as shown in FIG. 23, unsharp image generating means 57 forgenerating the unsharp luminance image data Lus and the unsharp colorimage data Cus which are unsharp image data of the luminance data L* andthe color data C* may be used instead of the multi-resolution conversionmeans 52, and the luminance histogram HLus and the color histogram HCusmay be generated from the unsharp luminance image data Lus and theunsharp color image data Cus respectively.

In addition, all of the contents of Japanese Patent Application Nos.11(1999)-230731 and 2000-173279 are incorporated into this specificationby reference.

1. An image processing method comprising the step of quantifying a senseof contrast of an image represented by image data, based on the imagedata, wherein the step of quantifying comprises the steps of: obtainingluminance data and color data representing luminance information andcolor information of the image from the image data; obtainingmulti-resolution luminance image data and/or multi-resolution colorimage data in a plurality of frequency bands by converting the luminancedata and/or the color data into multiple resolutions; generating aluminance histogram and/or a color histogram which are histograms of themulti-resolution luminance image data and/or the multi-resolution colorimage data in each of the frequency bands; and quantifying the sense ofcontrast based on the luminance histogram and/or the color histogram ineach of the frequency bands.
 2. An image processing method as claimed inclaim 1, wherein the step of quantifying comprises the steps of:generating unsharp image data of the image data; generating a histogramof the unsharp image data; and quantifying the sense of contrast basedon the histogram.
 3. An image processing method as claimed in claim 1,wherein the step of quantifying comprises the steps of: obtainingluminance data and color data representing luminance information andcolor information of the image from the image data; generating unsharpluminance image data and/or unsharp color image data which are unsharpimage data of the luminance data and/or the color data; generating aluminance histogram and/or a color histogram which are histograms of theunsharp luminance image data and/or the unsharp color image data; andquantifying the sense of contrast based on the luminance histogramand/or the color histogram.
 4. An image processing method as claimed inclaim 3, wherein the step of generating the luminance histogram and/orthe color histogram is the step of generating a color histogramrepresenting a two-dimensional frequency distribution of the unsharpcolor image data in the case where the color image data are generated.5. An image processing method as claimed in claim 1, wherein the step ofquantifying comprises the steps of: generating unsharp image data of theimage data; and quantifying the sense of contrast based on positioninformation of a light portion and/or a dark portion in an unsharp imagerepresented by the unsharp image data.
 6. An image processing methodcomprising the step of carrying out image processing for changingluminance information of an image represented by image data on the imagedata based on color information of the image, wherein the step ofcarrying out the image processing comprises the steps of: obtainingcolor data representing the color information from the image data;obtaining multi-resolution image data in a plurality of frequency bandsby converting the color data into multiple resolutions; generating ahistogram of multi-resolution image data in a lowermost frequency bandout of the multi-resolution image data in the plurality of frequencybands; and carrying out the image processing on the image data based onthe histogram.
 7. An image processing method as claimed in claim 6,wherein the step of carrying out the image processing comprises thesteps of: obtaining color data representing the color information fromthe image data; generating unsharp image data of the color data;generating a histogram of the unsharp image data; and carrying out theimage processing on the image data based on the histogram.
 8. An imageprocessing method as claimed in claim 6, wherein the step of generatingthe histogram is the step of generating a histogram representing atwo-dimensional frequency distribution of the unsharp image data.
 9. Animage processing method as claimed in claim 6, wherein the step ofquantifying comprises the steps of: setting a pattern for imageprocessing to be carried out on the image data based on the colorhistogram.
 10. An image processing apparatus comprising contrast-sensequantification means for quantifying a sense of contrast of an imagerepresented by image data, based on the image data, wherein thecontrast-sense quantification means comprises: conversion means forobtaining luminance data and color data representing luminanceinformation and color information of the image from the image data;multi-resolution conversion means for obtaining multi-resolutionluminance image data and/or multi-resolution color image data in aplurality of frequency bands by converting the luminance data and/or thecolor data into multiple resolutions; histogram generating means forgenerating a luminance histogram and/or a color histogram which arehistograms of the multi-resolution luminance image data and/or themulti-resolution color image data in each of the frequency bands; andquantification means for quantifying the sense of contrast based on theluminance histogram and/or the color histogram in each of the frequencybands.
 11. An image processing apparatus as claimed in claim 10, whereinthe contrast-sense quantification means comprises: unsharp image datagenerating means for generating unsharp image data of the image data;histogram generating means for generating a histogram of the unsharpimage data; and quantification means for quantifying the sense ofcontrast based on the histogram.
 12. An image processing apparatus asclaimed in claim 10, wherein the contrast-sense quantification meanscomprises: conversion means for obtaining luminance data and color datarepresenting luminance information and color information of the imagefrom the image data; unsharp image data generating means for generatingunsharp luminance image data and/or unsharp color image data which areunsharp image data of the luminance data and/or the color data;histogram generating means for generating a luminance histogram and/or acolor histogram which are histograms of the unsharp luminance image dataand/or the unsharp color image data; and quantification means forquantifying the sense of contrast based on the luminance histogramand/or the color histogram.
 13. An image processing apparatus as claimedin claim 12, wherein the histogram generating means generates a colorhistogram representing a two-dimensional frequency distribution of theunsharp color image data in the case where the unsharp image datagenerating means generates the unsharp color image data.
 14. An imageprocessing method as claimed in claim 10, wherein the contrast-sensequantification means comprises: unsharp image data generating means forgenerating unsharp image data of the image data; and quantificationmeans for quantifying the sense of contrast based on positioninformation of a light portion and/or a dark portion in an unsharp imagerepresented by the unsharp image data.
 15. An image processing apparatuswhich carries out image processing on image data for changing luminanceinformation of an image represented by the image data, based on colorinformation of the image, comprising: conversion means for obtainingcolor data representing the color information of the image from theimage data; multi-resolution conversion means for obtainingmulti-resolution image data in a plurality of frequency bands byconverting the color data into multiple resolutions; histogramgenerating means for generating a histogram of multi-resolution imagedata in a lowermost frequency band out of the multi-resolution imagedata in the plurality of frequency bands; and processing means forcarrying out the image processing on the image data based on thehistogram.
 16. An image processing apparatus as claimed in claim 10,comprising: conversion means for obtaining color data representing thecolor information of the image from the image data; unsharp imagegenerating means for generating unsharp image data of the color data;histogram generating means for generating a histogram of the unsharpimage data; and processing means for carrying out the image processingon the image data based on the histogram.
 17. An image processingapparatus as claimed in claim 16, wherein the histogram generating meansgenerates a histogram representing a two-dimensional frequencydistribution of the unsharp image data.
 18. An image processingapparatus as claimed in claim 15, wherein the step of quantifyingcomprises the step of: pattern setting means for setting a pattern forimage processing to be carried out on the image data based on the colorhistogram.
 19. A computer-readable recording medium storing a program tocause a computer to execute an image processing method comprising theprocedure of quantifying a sense of contrast of an image represented byimage data, based on the image data, wherein the procedure ofquantifying the sense of contrast comprises the procedures of: obtainingluminance data and color data representing luminance information andcolor information of the image from the image data; obtainingmulti-resolution luminance image data and/or multi-resolution colorimage data in a plurality of frequency bands by converting the luminancedata and/or the color data into multiple resolutions; generating aluminance histogram and/or a color histogram which are histograms of themulti-resolution luminance image data and/or the multi-resolution colorimage data in each of the frequency bands; and quantifying the sense ofcontrast based on the luminance histogram and/or the color histogram ineach of the frequency bands.
 20. A computer-readable recording medium asclaimed in claim 19, wherein the procedure of quantifying the sense ofcontrast comprises the procedures of: generating unsharp image data ofthe image data; generating a histogram of the unsharp image data; andquantifying the sense of contrast based on the histogram.
 21. Acomputer-readable recording medium as claimed in claim 19, wherein theprocedure of quantifying the sense of contrast comprises the proceduresof: obtaining luminance data and color data representing luminanceinformation and color information of the image from the image data;generating unsharp luminance image data and/or unsharp color image datawhich are unsharp image data of the luminance data and/or the colordata; generating a luminance histogram and/or a color histogram whichare histograms of the unsharp luminance image data and/or the unsharpcolor image data; and quantifying the sense of contrast based on theluminance histogram and/or the color histogram.
 22. A computer-readablerecording medium as claimed in claim 19, wherein the procedure ofgenerating the luminance histogram and/or the color histogram is thestep of generating a color histogram representing a two-dimensionalfrequency distribution of the unsharp image data in the case where thecolor image data are generated.
 23. A computer-readable recording mediumas claimed in claim 19, wherein the procedure of quantifying the senseof contrast comprises the procedures of: generating unsharp image dataof the image data; and quantifying the sense of contrast based onposition information of a light portion and/or a dark portion in anunsharp image represented by the unsharp image data.
 24. Acomputer-readable recording medium storing a program to cause a computerto execute an image processing method for carrying out image processingfor changing luminance information of an image represented by image dataon the image data, based on color information of the image, wherein theprogram comprises the procedures of: obtaining color data representingthe color information from the image data; obtaining multi-resolutionimage data in a plurality of frequency bands by converting the colordata into multiple resolutions; generating a histogram ofmulti-resolution image data in a lowermost frequency band out of themulti-resolution image data in the plurality of frequency bands; andcarrying out the image processing on the image data based on thehistogram.
 25. A computer-readable recording medium as claimed in claim24, wherein the program comprises the procedures of: obtaining colordata representing the color information from the image data; generatingunsharp image data of the color data; generating a histogram of theunsharp image data; and carrying out the image processing on the imagedata based on the histogram.
 26. A computer-readable recording medium asclaimed in claim 25, wherein the procedure of generating the histogramis the procedure of generating a histogram representing atwo-dimensional frequency distribution of the unsharp image data.
 27. Acomputer-readable recording medium as claimed in claim 24, wherein thestep of quantifying comprises the step of: setting a pattern for imageprocessing to be carried out on the image data based on the colorhistogram.